Operating AI by Ulrika Jagare

Operating AI by Ulrika Jagare

Author:Ulrika Jagare [Jägare, Ulrika]
Language: eng
Format: epub
ISBN: 9781119833215
Publisher: Wiley
Published: 2022-05-17T00:00:00+00:00


The ML Inference Pipeline

Machine learning (ML) inference is the second phase of the ML pipeline, in which the model is put into action on live data to produce actionable output.

According to a technical report (https://dancrankshaw.com/publication/inferline-pub) by Dan Crankshaw in InferLine in 2018, the dominant cost in production machine learning workloads is not training individual models but serving predictions from increasingly complex prediction pipelines spanning multiple models, machine learning frameworks, and parallel hardware accelerators. Due to the complex interaction between model configurations and parallel hardware, prediction pipelines are challenging to provision and costly to execute when serving interactive latency-sensitive applications. This challenge is exacerbated by the unpredictable dynamics of bursty workloads. From a single model perspective, however, model serving or inference is normally much less computationally intense than model training.

The setup of your inference pipeline therefore depends a lot on how you want to serve your ML system. There are multiple ways to serve your ML model, and often you are required to decide the serving architecture even before training your model. This means that this step should be part of the ML development phase. Why? Well, if you don't know how your ML system needs to operate in the live production environment, how will you know what to expose your model to during training?

So, what is a serving architecture for your ML system? It's simply the conceptual model that defines the structure, behavior, and more views of the ML system. A serving architecture description is a formal description and representation of the system, organized in a way that supports reasoning about the structures and behaviors of the system.

A serving architecture can consist of system components, and the subsystems developed, that will work together to implement the overall ML system. So, to select the best approach for your serving architecture, you need to know and consider the requirements for your ML system. For example:

What will ensure a good user experience? To answer this, you need to be clear on who the user of your solution is. Remember, the user could be an actual person, or a consumer (downstream service) such as a backend service.

What parts of the solution are the most critical to implement for a good user experience?

What is the minimum viable product you can implement as soon as possible to get the first version to your users and collect feedback?



Download



Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.
Popular ebooks
A Developer's Guide to Building Resilient Cloud Applications with Azure by Hamida Rebai Trabelsi(9394)
Distributed Machine Learning with Python by Guanhua Wang(3621)
Getting Started with CockroachDB by Kishen Das Kondabagilu Rajanna(2577)
Exploratory Data Analysis with Python Cookbook by Ayodele Oluleye(1417)
Getting Started With CockroachDB: A Guide to Using a Modern, Cloud-Native, and Distributed SQL Database for Your Data-Intensive Apps by Kishen Das Kondabagilu. Rajanna(1248)
R Web Scraping Quick Start Guide by Olgun Aydin(1083)
PostgreSQL 13 Cookbook: Over 120 recipes to build high-performance and fault-tolerant PostgreSQL database solutions by Vallarapu Naga Avinash Kumar(1016)
Mastering PostgreSQL 15 - Fifth Edition by Hans-Jürgen Schönig(691)
Pandas for Everyone: Python Data Analysis, 2nd Edition by Daniel Y. Chen(451)
Apache Hadoop 3 Quick Start Guide by Hrishikesh Karambelkar(450)
Learn SQL with MySQL: Retrieve and Manipulate Data Using SQL Commands with Ease by Ashwin Pajankar(407)
SQL Query Design Patterns and Best Practices by Steve Hughes & Dennis Neer & Dr. Ram Babu Singh & Shabbir H. Mala & Leslie Andrews & Chi Zhang(395)
Deploy Node.js on GCP: A comprehensive guide to deploying Node.js on Google Cloud Platform by Jonathan Lin(378)
Configuring Sales and Distribution in SAP ERP by Unknown(362)
Leveling Up with SQL by Mark Simon(338)
Learning Data Science by Sam Lau(326)
Intermediate Python by Oswald Campesato(322)
The Definitive Guide to Data Integration by Pierre-Yves BONNEFOY Emeric CHAIZE Raphaël MANSUY Mehdi TAZI(305)
Data Engineering with AWS: A Comprehensive Guide to Building Robust Data Pipelines by Paul Brian(299)
Pandas Basics by Oswald Campesato(296)